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ABSTRACT 

Visible,  infrared  (IR)  and  sensor-fused  imagery  of  scenes  that  contain  occluded  camouflaged  threats  are  compared  on  a 
two  dimensional  (2D)  display  and  a  three  dimensional  (3D)  display.  A  3D  display  is  compared  alongside  a  2D  monitor 
for  hit  and  miss  differences  in  the  probability  of  detection  of  objects.  Response  times  are  also  measured.  Image  fusion  is 
achieved  using  a  Gaussian  Laplacian  pyramidal  approach  with  wavelets  for  edge  enhancement.  Detecting  potential 
threats  that  are  camouflaged  or  difficult  to  see  is  important  not  only  for  military  acquisition  problems  but,  also  for  crowd 
surveillance  as  well  as  tactical  use  such  as  on  border  patrols.  Imaging  and  display  technologies  that  take  advantage  of 
3D  and  sensor  fusion  will  be  discussed. 


1.  INTRODUCTION 

Computer  driven  interactive  3D  imaging  has  made  significant  progress  during  the  past  decade  and  3D  display  technology 
continues  to  develop  rapidly.  One  of  the  important  applications  of  this  technology  is  an  interactive  tool  for  vehicle 
design.  The  Zebra  Imaging  Company  in  Austin,  Texas  investigates  technical  feasibility  of  developing  such  a  toolset 
using  a  holoprinter  and  integral  imaging  concepts.  The  largest  in  the  world  hologram  of  a  Ford  car,  which  was  generated 
by  Zebra  Imaging  using  a  holoprinter,  was  displayed  at  the  Detroit  Autoshow  in  1998-1999.  The  size  of  this  hologram 
was  6’  x  4’.  The  disadvantage  of  this  holoprinter  was  its  slow  speed.  It  took  several  days  to  print  out  a  computer 
generated  hologram  (CGH)  of  a  vehicle.  The  Zebra  Imaging  was  utilizing  a  CAD  image  of  a  Ford  vehicle  as  an  input  for 
CGH.  Defense  Evaluation  and  Research  Agency  (DERA)  in  the  UK  is  developing  a  3D  display  for  vehicle  design, 
which  is  interactive  in  real  time.  It  is  based  on  computer  generated  Fourier  holograms  (CGFH).  DERA  is  implementing 
a  high-resolution  LCD  as  an  output  device  for  an  interactive  in  real  time  volumetric  image  of  a  vehicle.  The  drawback  of 
the  3D  holographic  real  time  display  was  its  computational  intensity.  The  computation  of  CGFH  of  a  3D  object  with 
outside  dimensions  of  2”  x  2”  x  2”,  its  coding  and  reconstruction  in  real  time  required  20  billion  computations.  An 
alternative  approach  is  a  stereoscopic  display,  utilizing  a  3D  CAD  package  as  an  input  image,  for  example,  IcemSurf, 
implemented  by  Ford  Motor  Company.  A  haptic  interface  is  used  for  an  output  image  for  working  in  3D.  This  image 
can  be  viewed  using  stereoscopic  glasses  or  a  heads-up  display.  The  review  of  the  3D  imaging  technologies  can  be 
found  in  the  reference  work  by  McCallister  [1], 

The  Visual  Perception  Laboratory  (VPL)  at  TACOM  is  interested  in  exploring  various  display  technologies  for 
applications  having  to  do  with  camouflage  assessment  of  military  vehicles,  non-destructive  testing,  medical  telepresence 
and  homeland  defense.  The  realism  and  depth  perception  provided  by  3D  displays  is  important  to  photo  simulation  tests 
having  to  do  with  camouflage  because  it  is  important  to  know  how  close  combat  vehicles  can  get  to  the  enemy  without 
being  detected.  At  present,  the  TARDEC  VPL  has  a  1 80-degree  wrap  around  screen  and  three  high  resolution  projectors 
that  provide  imagery  with  pixels  that  subtend  less  than  1  arc  second  at  15  to  16  feet  from  the  screen.  The  VPL  team  is 
interested  in  implementing  new  3D  display  technologies  with  improved  resolution  and  color  fidelity. 
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The  authors  were  able  to  overcome  some  difficulties  associated  with  holographic  displays  and  the  inconvenience  of 
using  stereoscopic  glasses  or  heads-up  display,  by  implementing"  an  autostereoscopic  display  system,  VIS4D™, 
developed  by  Ethereal  Technologies,  Inc.,  of  Ann  Arbor,  Michigan.  The  core  component  of  this  system,  an  optical 
platform  (Figure  3),  is  complemented  by  Stretchable  Membrane  Mirror  (SMM)  technology  [2]  developed  at  the 
University  of  Strathclyde  in  Glasgow,  Scotland.  This  large  format,  light  weight,  variable  focal  length  mirror  is  a  cheap 
alternative  to  conventional  fixed-curvature,  heavy  weight,  expensive  glass  based  optics.  [2], 

2.  EXPERIMENTAL  PROCEDURE 


Fourteen  civilian  subjects  were  picked  randomly  from  the  employee  population  at  TARDEC.  Each  subject  took  both  the 
2D  test  and  3D  test.  Which  test  they  took  first  was  determined  by  a  random  number  generator.  The  subjects  were  then 
evaluated  to  verify  if  they  could  experience  stereopsis,  and  were  also  shown  how  to  use  the  VIS4D  workstation.  They 
were  shown  how  to  align  themselves  by  using  a  rangefinder  made-up  of  two  orange  disks  placed  at  the  top  of  mirror 
assembly.  They  were  shown  stereo  pairs  and  asked  to  verify  they  could  see  a  3D  image.  Once  they  were  comfratable 
with  the  overall  environment  the  test  began. 

The  photo  simulation  test  consisted  of  ten  images  in  a  random  order.  The  2D  and  3D  test  had  the  same  images,  albeit  the 
3D  test  used  stereo  pairs.  The  2D  test  was  done  on  a  flat  panel  monitor  and  the  3D  test  was  done  on  the  VIS4D 
workstation.  Each  image  had  four  objects  defined  and  labeled  A,  B,  C,  and  D.  The  purpose  of  the  test  was  to  determine 
the  relative  order  of  the  four  objects  from  closest  through  the  farthest  starting  with  the  closest  object  and  ending  with  the 
farthest  object.  There  was  no  time  limit,  but  it  took  the  subjects  about  5-10  minutes  to  take  the  test.  Response  time  was 
measured  during  the  test  for  each  subject  and  each  target  in  the  image. 


Fig.’s  1,  2,  and  3  below  show  various  details  of  the  VIS4D  workstation.  In  Fig.  1,  one  can  see  that  the  footprint  mirror 
assembly  is  2.7  M  x  1.6  M  and  the  height  of  the  mirror  assembly  is  also  1.6  M.  The  image  occupies  most  the  field  of 
view  of  a  subject.  The  larger  viewing  format  assists  through  the  augmented  realty  the  effect  of  immersion  that  an 
observer  can  experience  while  using  system. 

Fig.  2  below  shows  the  workstation  desk  that  is  1.5  M  from  the  front  of  the  display.  The  desk  provides  a  comfortable 
place  for  the  user  to  work  and  make  decisions. 


Fig.  1:  V1S4D™  system  with  desk 


Fig.  3:  Schematic  of  optical  platform  with  SMM 


3.  THE  UTILITY  VOLUMETRIC  DISPLAYS 


Cognitive  systems  represent  a  new  rapidly  developing  area  of  technology.  The  term  cognitive  system  is  referring  to 
systems  and  processes  similar  to  the  human  mind  such  as  understanding,  reasoning,  judgment,  memory  and  learning. 

Joint  cognitive  systems  combine  computer  and  human  system  components  contributing  to  the  performance  of  system 
cognitive  functions.  Information  design  can  influence  the  nature  of  communication  through  the  form  of  presentation  of 
displayed  information.  Cognitive  displays  facilitate  information  presentation  through  organization  in  relation  to  the  tasks 
to  be  performed.  Cognitive  imagery  displays  can  present  the  appearance  of  a  3D  electronic  window  of  the  world  to  make 
the  imagery  more  intuitive,  cognitively  compatible,  and  immediately  task  relevant.  This  can  include  functions  such  as 
electronic  cueing  and  highlighting  of  camouflaged,  hidden  or  partially  obscured  objects.  The  objects  of  interest  can  be 
determined  directly  from  the  visual  scene.  Automated  target  recognition  (ATR)  function  can  be  implemented  with  image 
processing  and  displays. 

A  display  can  be  defined  as  a  structured  and  purposive  presentation  of  information  to  the  human  senses  [Stokes,  Wickens 
&  Kite,  1990].  The  function  of  a  display  is  to  support  a  user  in  achieving  an  objective  by  providing  essential 
information.  Displays  may  access  more  information  than  is  available  to  direct  observation  by  an  operator.  The  limits  of 
an  operator’s  cognitive  resources  constrain  the  level  of  performance  that  can  be  achieved. 

Perception  has  been  defined  as  the  functional  transformation  of  sensory  data  into  Situational  Awareness  (SA).  SA 
describes  the  perception  of  elements  in  an  environment  within  a  volume  of  time  and  space,  the  comprehension  of  their 
meaning,  and  the  projection  of  their  status  in  the  near  future.  Spatial  awareness  is  an  important  component  of  SA.  It  is 
an  operator’s  perceptual  and  cognitive  comprehension  of  the  3D  geometry  of  the  environment  in  which  he  or  she  is 
operating.  [3] 

It  is  important  to  mention  some  advantages  and  disadvantages  of  the  implementation  of  3D  displays.  Three-dimensional 
displays  provide  more  natural  representation  of  the  visual  scene  of  the  real  world;  they  reduce  the  need  for  mental 
integration  of  information  from  multiple  sources.  Tasks  in  which  spatial  awareness  is  critical  may  be  facilitated  by  the 
use  of  3D  technology.  The  disadvantages  of  3D  representation  are;  it  can  effect  the  viewer’s  ability  to  make  precise 
judgments  on  each  dimension  alone;  it  creates  some  ambiguity  regarding  the  precise  location  of  objects  along  the  line  of 
sight.  In  other  words,  small  artifacts  in  the  design  of  the  mirror  can  potentially  cause  some  ambiguity.  These  factors 
should  be  taken  into  consideration  while  making  a  decision  in  selecting  a  display  system. 

4.  DATA 

Fig.  4  below  shows  the  task  the  viewers  were  given,  namely,  to  determine  the  relative  range  of  the  targets  in  the  image 
using  the  2D  monitor  and  the  3D  display.  The  vehicles  were  labeled  A,B,C,  and  D  and  subjects  were  asked  to  rank  the 
vehicles  according  to  increasing  range.  The  baseline  from  which  distance  is  determined  is  the  curb  in  the  image. 


Fig.  4:  Example  image  from  the  test 


Test  Data  Analysis 

The  chart  below  in  Fig.  5  shows  the  number  of  correct  responses  for  all  subjects  summed  over  all  pictures.  The  graph 
shows  that  the  performance  is  highly  dependent  on  the  subject.  This  difference  may  in  part  be  due  to  the  need  of  the 
subjects  to  situate  themselves  in  the  eyebox. 


Fig.  5:  Overall  response  lime  for  2D  versus  3D  displays 


STATISTICAL  ANALYSIS 

We  are  interested  in  determining  whether  viewing  outdoor  scenes  with  a  3-D  display  improves  depth  perception  over 
viewing  the  same  outdoor  scene  with  a  2-D  display.  Each  subject  was  shown  ten  outdoor  scenes  in  a  random  order  and 
asked  to  rank  four  targets  from  closest  to  farthest.  The  percentage  of  correct  responses  was  recorded. 

Since  subjects  differ  in  their  perception  skills,  we  shall  use  them  as  blocks.  Once  a  subject  is  chosen,  the  order  in  which 
type  of  display  is  presented  is  randomly  determined.  There  was  a  several  week  interval  between  tests  to  eliminate  any 
learning  from  the  first  test.  Thus,  we  have  a  randomized  complete  block  design.  The  complete  analysis  variance  for  this 
experiment  is  summarized  in  Table  1.  Both  DISPLAY  and  SUBJECT  are  not  significant.  Thus,  we  conclude  that  the 
display  type  does  not  affect  the  subject’s  score  and  that  there  is  not  a  significant  difference  between  subjects. 


Tests  of  Between-Subjects  Effects 


Dependent  Variable:  SCORE 


A  3D  display  could  also  be  applied  to  crowd  surveillance  and  screening  at  public  access  points.  The  images  above  in 
Fig.’s  1 1,  12,  and  13,  are  the  visual,  passive  millimeter  wave  [3],  and  fused  images  respectively  of  a  man  with  a 
concealed  weapon.  In  these  times  of  increased  concern  about  terrorists  and  passengers  carrying  concealed  weapons, 
sensor  fusion  and  3D  displays  could  be  of  benefit  in  alerting  guards  to  potential  problem  passengers.  Combing  the  sensor 
fusion  with  a  3D  display  could  also  improve  the  recognition  rate  of  guards  using  cameras  that  scan  crowds  for  people 
that  are  listed  in  a  known  terrorist  database. 

CONCLUSIONS 

The  authors  performed  a  test  in  which  the  response  time  and  the  number  of  correct  decisions  on  the  relative  range  of 
many  targets  is  compared  for  a  2D  versus  3D  display.  The  authors  found  that  the  recorded  response  time  was  greater 
using  the  3D  display,  and  there  was  no  significant  difference  in  the  number  of  correct  target  identifications  for  this  data 
set,  in  the  2D  versus  3D  display.  Training  and  experience  using  the  mirror  may  have  had  an  impact  on  the  performance 
of  the  subjects,  although,  the  test  monitors  tried  to  be  consistent  in  this  regard.  Another  factor  that  may  have  influenced 
the  test  was  the  degree  of  3D  perspective  in  the  images  that  were  used  for  the  test.  It  should  be  noted  that  when  people 
view  demonstrations  in  the  laboratory,  there  is  agreement  that  a  true  3D  image  can  be  seen  in  the  mirror. 

Given  this  was  the  first  attempt  to  compare  2D  versus  3D  here  at  TARDEC,  this  was  a  learning  experience.  Images 
within  the  stereo  pairs  contained  alignment  and  nomenclature  problems  that  effected  stereopsis  experienced  by  the 
subjects,  as  reflected  in  the  statistics.  Subsequent  test  are  planned  where  these  issues  will  be  addressed  and  corrected. 

Future  areas  for  research  include  testing  the  VIS4D  workstation  in  remote  telemedicine  applications  and  non-destructive 
material  testing  using  volumetric  imaging  technologies. 
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